Dimension reduction-based significance testing in nonparametric regression

A dimension reduction-based adaptive-to-model test is proposed for significance of a subset of covariates in the context of a nonparametric regression model. Unlike existing local smoothing significance tests, the new test behaves like a local smoothing test as if the number of covariates were just that under the null hypothesis and it can detect local alternatives distinct from the null at the rate that is only related to the number of covariates under the null hypothesis. Thus, the curse of dimensionality is largely alleviated when nonparametric estimation is inevitably required. In the cases where there are many insignificant covariates, the improvement of the new test is very significant over existing local smoothing tests on the significance level maintenance and power enhancement. Simulation studies and a real data analysis are conducted to examine the finite sample performance of the proposed test.Comment: 49 pages, 2 figure

Specification testing for regressions: an approach bridging between local smoothing and global smoothing methods

For regression models, most of existing specification tests can be categorized into the class of local smoothing tests and of global smoothing tests. Compared with global smoothing tests, local smoothing tests can only detect local alternatives distinct from the null hypothesis at a much slower rate when the dimension of predictor vector is high, but can be more sensitive to oscillating alternatives. In this paper, we suggest a projection-based test to bridge between the local and global smoothing-based methodologies such that the test can benefit from the advantages of these two types of tests. The test construction is based on a kernel estimation-based method and the resulting test becomes a distance-based test with a closed form. The asymptotic properties are investigated. Simulations and a real data analysis are conducted to evaluate the performance of the test in finite sample cases.Comment: 31 page

Integrated conditional moment test and beyond: when the number of covariates is divergent

The classic integrated conditional moment test is a promising method for testing regression model misspecification. However, it severely suffers from the curse of dimensionality. To extend it to handle the testing problem for parametric multi-index models with diverging number of covariates, we investigate three issues in inference in this paper. First, we study the consistency and asymptotically linear representation of the least squares estimator of the parameter matrix at faster rates of divergence than those in the literature for nonlinear models. Second, we propose, via sufficient dimension reduction techniques, an adaptive-to-model version of the integrated conditional moment test. We study the asymptotic properties of the new test under both the null and alternative hypothesis to examine its ability of significance level maintenance and its sensitivity to the global and local alternatives that are distinct from the null at the fastest possible rate in hypothesis testing. Third, we derive the consistency of the bootstrap approximation for the new test in the diverging dimension setting. The numerical studies show that the new test can very much enhance the performance of the original ICM test in high-dimensional scenarios. We also apply the test to a real data set for illustrations

Adaptive-to-model hybrid of tests for regressions

In model checking for regressions, nonparametric estimation-based tests usually have tractable limiting null distributions and are sensitive to oscillating alternative models, but suffer from the curse of dimensionality. In contrast, empirical process-based tests can, at the fastest possible rate, detect local alternatives distinct from the null model, but is less sensitive to oscillating alternative models and with intractable limiting null distributions. It has long been an issue on how to construct a test that can fully inherit the merits of these two types of tests and avoid the shortcomings. We in this paper propose a generic adaptive-to-model hybrid of moment and conditional moment-based test to achieve this goal. Further, a significant feature of the method is to make nonparametric estimation-based tests, under the alternatives, also share the merits of existing empirical process-based tests. This methodology can be readily applied to other kinds of data and constructing other hybrids. As a by-product in sufficient dimension reduction field, the estimation of residual-related central subspace is used to indicate the underlying models for model adaptation. A systematic study is devoted to showing when alternative models can be indicated and when cannot. This estimation is of its own interest and can be applied to the problems with other kinds of data. Numerical studies are conducted to verify the powerfulness of the proposed test.Comment: 35pages, 6figure

Model checking for generalized linear models: a dimension-reduction model-adaptive approach

Local smoothing testing that is based on multivariate nonparametric regression estimation is one of the main model checking methodologies in the literature. However, relevant tests suffer from the typical curse of dimensionality resulting in slow convergence rates to their limits under the null hypotheses and less deviation from the null under alternatives. This problem leads tests to not well maintain the significance level and to be less sensitive to alternatives. In this paper, a dimension-reduction model-adaptive test is proposed for generalized linear models. The test behaves like a local smoothing test as if the model were univariate, and can be consistent against any global alternatives and can detect local alternatives distinct from the null at a fast rate that existing local smoothing tests can achieve only when the model is univariate. Simulations are carried out to examine the performance of our methodology. A real data analysis is conducted for illustration. The method can readily be extended to global smoothing methodology and other testing problems

Estimation and adaptive-to-model testing for regressions with diverging number of predictors

The research described in this paper is motivated by model checking for parametric single-index models with diverging number of predictors. To construct a test statistic, we first study the asymptotic property of the estimators of involved parameters of interest under the null and alternative hypothesis when the dimension is divergent to infinity as the sample size goes to infinity. For the testing problem, we study an adaptive-to-model residual-marked empirical process as the basis for constructing a test statistic. By modifying the approach in the literature to suit the diverging dimension settings, we construct a martingale transformation. Under the null, local and global alternative hypothesis, the weak limits of the empirical process are derived and then the asymptotic properties of the test statistic are investigated. Simulation studies are carried out to examine the performance of the test

An adaptive-to-model test for partially parametric single-index models

Residual marked empirical process-based tests are commonly used in regression models. However, they suffer from data sparseness in high-dimensional space when there are many covariates. This paper has three purposes. First, we suggest a partial dimension reduction adaptive-to-model testing procedure that can be omnibus against general global alternative models although it fully use the dimension reduction structure under the null hypothesis. This feature is because that the procedure can automatically adapt to the null and alternative models, and thus greatly overcomes the dimensionality problem. Second, to achieve the above goal, we propose a ridge-type eigenvalue ratio estimate to automatically determine the number of linear combinations of the covariates under the null and alternatives. Third, a Monte-Carlo approximation to the sampling null distribution is suggested. Unlike existing bootstrap approximation methods, this gives an approximation as close to the sampling null distribution as possible by fully utilising the dimension reduction model structure under the null. Simulation studies and real data analysis are then conducted to illustrate the performance of the new test and compare it with existing tests.Comment: 35 pages, 2 figure

A projection-based adaptive-to-model test for regressions

A longstanding problem of existing empirical process-based tests for regressions is that when the number of covariates is greater than one, they either have no tractable limiting null distributions or are not omnibus. To attack this problem, we in this paper propose a projection-based adaptive-to-model approach. When the hypothetical model is parametric single-index, the method can fully utilize the dimension reduction model structure under the null hypothesis as if the covariate were one-dimensional such that the martingale transformation-based test can be asymptotically distribution-free. Further, the test can automatically adapt to the underlying model structure such that the test can be omnibus and thus detect alternative models distinct from the hypothetical model at the fastest possible rate in hypothesis testing. The method is examined through simulation studied and is illustrated by a real data analysis

Bounds smaller than the Fisher information for generalized linear models

In this paper, we propose a parameter space augmentation approach that is based on "intentionally" introducing a pseudo-nuisance parameter into generalized linear models for the purpose of variance reduction. We first consider the parameter whose norm is equal to one. By introducing a pseudo-nuisance parameter into models to be estimated, an extra estimation is asymptotically normal and is, more importantly, non-positively correlated to the estimation that asymptotically achieves the Fisher/quasi Fisher information. As such, the resulting estimation is asymptotically with smaller variance-covariance matrices than the Fisher/quasi Fisher information. For general cases where the norm of the parameter is not necessarily equal to one, two-stage quasi-likelihood procedures separately estimating the scalar and direction of the parameter are proposed. The traces of the limiting variance-covariance matrices are in general smaller than or equal to that of the Fisher/quasi-Fisher information. We also discuss the pros and cons of the new methodology, and possible extensions. As this methodology of parameter space augmentation is general, and then may be readily extended to handle, say, cluster data and correlated data, and other models.Comment: 32 page

A robust adaptive-to-model enhancement test for parametric single-index models

In the research on checking whether the underlying model is of parametric single-index structure with outliers in observations, the purpose of this paper is two-fold. First, a test that is robust against outliers is suggested. The Hampel's second-order influence function of the test statistic is proved to be bounded. Second, the test fully uses the dimension reduction structure of the hypothetical model and automatically adapts to alternative models when the null hypothesis is false. Thus, the test can greatly overcome the dimensionality problem and is still omnibus against general alternative models. The performance of the test is demonstrated by both Monte Carlo simulation studies and an application to a real dataset